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DETAILED ACTION 

1. A request for continued examination under 37 CFR 1.1 14, including the fee set forth in 37 CFR 1.17(e), 
was filed in this application after final rejection. Since this application is eligible for continued examination under 
37 CFR 1.1 14, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.1 14. Applicant's submission filed on 19 December 2007 has been 
entered. 

2. Claims 1-28 have been presented for examination. 

Response to Arguments 

3. In view of Applicant's amendments, the prior art rejections of claims 15-28 have been withdrawn. 

4. Applicant's arguments with respect to claims 1-14 have been considered but are not persuasive. 

• i. Applicants submit, on page 1 5 of the remarks, that Ralston does not disclose the amended 
limitations, including a space system model. 

Examiner notes that the term "space system" is a vague term that is not defined by the 
specification. The specification states that as an example, the system being modeled can be 
that of the earth and the surrounding satellites. However, the limitations do not require this 
specific model — the claims state that satellite model status data and a space system model 
must be included in the system. The environment modeled by Ralston is interpreted to be 
analogous to the space system in the claim limitations. Ralston discloses a visualization 
system or method for developing a space system utilizing a space system model of the virtual 
world (figure 9) and a representation of satellite model status data where the representation 
includes satellite orbit (|0090]-[0091| satellite data, including the direction of its 
movement). Ralston discloses the above system and method where a viewer can select a 
satellite to view its satellite model status data (|0090|-|0091 1) and direct the satellite to move 
to a different position (|0061 1 discloses the moving of virtual objects, which include 
satellites |0090|). 
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Claim Re fections - 35 USC S 112 

The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more.claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

5. Claims 1-28 are rejected under 35 U.S.C. 112, second paragraph, as being indefinite for failing to 
particularly point out and distinctly claim the subject matter which applicant regards as the invention. Regarding 
claims 1, 8, 15 and 27, the terms "satellite model status data", "space system" and "space system model" are 
indefinite. Claims 1 and 8 recite possible representations of the status data, but it is unclear how this data is obtained. 
Is this empirical data? Claims 1 5 and 27 do not define the term nor do they clarify how the data is obtained. The 
claims do not explicitly recite the inclusion of a satellite model in either the virtual or real world, but they recite 
satellite model status data. It is unclear which satellites the status data are referring to. Regarding claim 1, the 
limitation "an output portion. ..configured to allow the viewer to develop a space system" is indefinite. If this is an 
output portion, how can it be used to allow the user to develop the system? Regarding claim 8, the limitation 
"developing a space system" is indefinite. What does this entail? How is it different from the space system model 
that is determined earlier in the claim? Regarding claim 15, the first limitation indicates that the virtual world that is 
displayed to the user already includes satellite model status data, but the subsequent limitation recites "selecting a 
satellite, by the viewer, to view satellite model status data of the satellite." Is this status data different from the 
previously presented status data? Examiner, notes that while "satellite model status data" is included in the 
representation, the satellites themselves are not necessarily included. Thus, there is no antecedent basis for the 
selecting and directing of satellites. The phrases "selecting a satellite, by the viewer" and "directing the satellite, by 
the viewer" are indefinite. Regarding claim 27, the third liiratation indicates that the virtual world that is displayed 
to the user already includes satellite model status data, but the subsequent limitation recites "allow the viewer to 
select a satellite to view satellite model status data of the satellite" Is this status data different from the previously 
presented status data? Examiner notes that while "satellite model status data" is included in the representation, the 
satellites themselves are not necessarily included. Thus, there is not antecedent basis for the selecting and directing 
of satellites. The limitation "an output portion.. .configured to allow the viewer to develop a space system" is 
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indefinite. If this is an output portion, how can it be used to allow the user to develop the system? All other claims 
are rejected by virtue of their dependency. 

Claim Rejections - 35 USC 6 101 

35 U.S.C. I0l reads as follows: ■ 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new 
and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. 

6. Claims 1-7 and 27-28 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non- 
statutory subject matter. Regarding claims 1 and 27, the claims are directed to a system, but appear to be comprised 
solely of software elements. All other claims are rejected by virtue of their dependency. 

Claim Rejections -35 USC S 102 
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the 
rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the invention was described in (l) an application for patent, published under section 122(b), by another filed in the 
United States before the invention by the applicant for patent or (2) a patent granted on an application for patent by 
another filed in the United States before the invention by the applicant for patent, except that an international 
application filed under the treaty defined in section 351(a) shall have the effects for purposes of this subsection of an 
application filed in the United States only if the international application designated the United States and was 
published under Article 21(2) of such treaty in the English language 

7. Claims 1, 3-5, 7-9, 10-12, 14-15, 17-18, 21-25 and 27-28 are rejected under 35 U.S.C. 102(e) as being 
clearly anticipated by Ralston (US Pub. No. 2003/0014212). 

Regarding claims 1 and 27: 

Ralston discloses a visualization system for developing a space system (the environment modeled by 
Ralston is interpreted to be analogous to the space system in the claim limitations) comprising: 

a. a positioning portion (figure 4a head position 405 renderer 400; |0057|) configured to 

determine a position of a viewer with respect to a real world (figure 7; |0068|) and a position of 
the viewer with respect to a virtual world (figure 8; |0070|), the positioning portion configured to 
allow the viewer to interact with the virtual world (figure 14; [0076]) 
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b. a modeling portion configured to specify the virtual world in response to a space system model of 
the virtual world (figure 1 Tenderer 400; |0057|) 

c. a model specification portion configured to specify a representation of satellite status data in 
response to the position of the viewer with respect to the virtual world and in response to the 
satellite status data (figure 25; |0090)-|0091|) 

d. an output portion configured to provide an image of the virtual super-imposed on an image of the 
real world (figure 25), and configured to allow the view to develop a pace system ([0061]; [0076] 
can alter the location of virtual objects), the image of the virtual world including the 
representation of the satellite model status data to the viewer in response to the position of the 
viewer with respect to the virtual world (figure 25; |0090]-|0091]; figure 27 output to 465) 

e. wherein the representation of the satellite data comprises a representation of the satellite orbit 
([0090] direction satellite is moving in ). 

Regarding claim 27, the "object" in this claim is equivalent to the "satellite" in claim 1 . Ralston discloses 
an input portion configured to allow the viewer to select a satellite to view satellite model status data of the 
satellite (|0090]-[0091| viewer can choose to view satellites and data) and configured to allow the viewer 
to direct the satellite to move to a different position ([0061 ] discloses the moving of virtual objects, 
which include satellites [0090|). 

Regarding claims 3 and 10: 

Ralston discloses the visualization system of claim 1 wherein the model of the virtual world is a multi- 
dimensional model of the virtual world ([0100]), and the output portion in a multi-dimensional output portion 
(|0062| real-world output presented in form of see through glasses, which is 3D). 

Regarding claims 4, 11, 17 and 28: 

Ralston discloses the visualization system of claim 1 wherein the representation of the satellite model 
status data comprises a representation of the current position (figure 25; ]0038|). 
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Regarding claims 5, 12 and 18: 

Ralston discloses the visualization system of claim 1 wherein the three-dimensional output portion 
comprises a heads-up pair of glasses (figure 4B headset 465; (00581; figure 9; |0071|). 

Regarding claims 7 and 14: 

Ralston discloses the visualization system of claim 5 wherein the heads-up pair of glasses area also 
configured to allow the viewer to view the image of the virtual world super-imposed on the image of the real world 
([0057] real world field of view augmented with virtual objects). 

Regarding claim 8: 

Ralston discloses a method for visualization of augmented reality to develop a space system, the method 
comprising: 

a. determine a position of a viewer with respect to a real world (figure 7; [0068]) and a position of 
the viewer with respect to a virtual world (figure 8; [0070J), the positioning portion configured to 
allow the viewer to interact with the virtual world (figure 14; |0076|) 

b. determining a space system model of the virtual world (figure 7 steps 720 and 725). The 
environment modeled by Ralston is interpreted to be analogous to the space system in the claim 
limitations. 

c. specify a representation of satellite model status data in response to the position of the viewer with 
respect to the virtual world and in response to the satellite model status data (figure 25; |0090|- 
|0091 1) 

d. displaying to the viewer a representation of the virtual world super-imposed on a representation of 
the real world ([0057) real world field of view augmented with virtual objects), the 
representation of the virtual world including the representation of the satellite status data in 
response to the position of the viewer with respect to the virtual world (figure 25; |0090|-[0091|). 

e. developing a space system (|0090|-|0091 1). This is interpreted to be analogous to the final model 
that is rendered and displayed to the user. 
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f. wherein the representation of the satellite data comprises a representation of the satellite orbit 
([0090] direction satellite is moving in ). 

Regarding claim 15: 

Ralston discloses a visualization method for developing a space system (the environment modeled by 
Ralston is interpreted to be analogous to the space system in the claim limitations) comprising: 

a. displaying to a viewer a representation of a real world overlaid with a representation of a virtual 
world ([0057] real world field of view augmented with virtual objects), the representation of 
the virtual world including the representation of the satellite model status data (figure 25; [0090]- 
|0091|). 

b. selecting a satellite, by the viewer, to view satellite model status data of the satellite (|0090|- 
[0091 1 operator can choose to view satellites and their status data) 

c. directing the satellite, by the viewer, to move to a different position ([0061 ] discloses the moving 
of virtual objects, which include satellites |0090|) 

d. wherein the representation of the virtual world is determined in response to a space system model 
of the virtual world (figure 7 steps 720 and 725) and in response to a position of the viewer with 
respect to the virtual world (figure 7; [0068]; figure 8; [00701). 

e. wherein the representation of the satellite model status data is determined in response to satellite 
model status data, and in response to a position of the viewer with respect to the virtual world 
(figure 25; [0090|-|0091 1). 

f. wherein the viewer is allowed to interact with the virtual world (figure 14; |0076|) 

Regarding claims 21-23: 

Ralston discloses providing the representation of the real world and the virtual world (and super-imposing 
the virtual world on the real world) in real time ([0055]; [0057]). 



Regarding claim 24: 
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Ralston discloses allowing the viewer to select a satellite or geographic area ([0091 1). 



Regarding claim 25: 

Ralston discloses allowing the viewer to directly select and manipulate objects in the virtual world without 
using a mouse ([0061 1). 

Claim Rejections - 35 USC 6 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set 
forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 
102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the 
subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill 
in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 



The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. ), 148 USPQ 459 (1966), that are 
applied for establishing a background for determining obviousness under 35 U.S.C. 103(a) are summarized as 
follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 

This application currently names joint inventors. In considering patentability of the claims under 35 
U.S.C. 103(a), the examiner presumes that the subject matter of the various claims was commonly owned at the time 
any inventions covered therein were made absent any evidence to the contrary. Applicant is advised of the 
obligation under 37 CFR 1 .56 to point out the inventor and invention dates of each claim that was not commonly 
owned at the time a later invention was made in order for the examiner to consider the applicability of 35 
U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) prior art under 35 U.S.C. 103(a). 

8. Claims 2, 6, 9, 13, 16, 19, 20 and 26 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ralston (US Pub. No. 2003/0014212) in view of Kato ("Virtual Object Manipulation on a Table-Top AR 
Environment", 2000). 



Regarding claims 2, 9 and 16: 
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Ralston does not explicitly disclose the use of pre-determined markers in the real world. Kato teaches an 
augmented reality system including: 

a. an image acquisition source configured to capture at least an image comprising an image of the 
real world (Kato: section 3 4 th paragraph "users. ..see the real world through the video 
camera; section 4.1), and an image of at least a pre-determined marker positioned in the real 
world (Kato: section 3 4 ,h -5 ,h paragraphs: real cards labeled with tracking markers, and 
computer vision techniques are used to find the tracking mark) 

b. an image processing portion configured to determine the position of the viewer with respect to the 
read world in response to the image of the pre-determined marker (s Kato: section 3 4 th 
paragraph determine the exact pose of the head mounted camera relative to the tracker; 
section 4.1) 

c. virtual positioning portion configured to translate the position of the viewer in the real world to the 
position of the viewer in the virtual world (Kato: section 3 4 th paragraph once the position of 
the real camera is known, virtual image can be exactly overlaid on it; section 4.1) 

At the time of the invention, it would have been obvious to one of ordinary skill in the arts to combine the 
teachings of Ralston and Kato because the interface of Kato provides accurate tracking and registration techniques 
and an intuitive and useful interface (Kato: abstract). 

Regarding claims 6 and 13: 

Kato teaches the visualization system of claim 2 wherein the image acquisition source (a video camera) is 
physically coupled to the heads-up pair of glasses (section 3 1 st paragraph "people. ..wear Olympus HMDs with 
cameras attached"). 

Regarding claim 19: 

Ralston does not explicitly disclose the use of pre-determined markers in the real world. Kato teaches an 
augmented reality system including displaying to the viewer a portion of the virtual selected by the viewer wherein 
the viewer selection is determined in response to a position of the a viewer-controlled marker with respect to the 
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virtual world, wherein the marker is positioned in the real world (Kato: section 3). At the time of the invention, it 
would have been obvious to one of ordinary skill in the arts to combine the teachings of Ralston and Kato because 
the interface of Kato provides accurate tracking and registration techniques and an intuitive and useful interface 
(Kato: abstract). 

Regarding claim 20: 

Kato discloses the visualization method of claim 19, wherein the step of displaying to the viewer the 
portion of the virtual world selected by the viewer comprises overlaying an icon over the portion of the virtual world 
displayed to the viewer (Kato: section 3). 

Regarding claim 26: 

Ralston does not explicitly disclose the use of pre-determined markers in the real world. Kato teaches an 
augmented reality system including a marker positioned in the real world, the marker is static or placed upon a 
paddle that included a pre-defined visual marker in the real world, and the paddle is capable of being moved around 
the real world (Kato: sections 3 and 4). At the time of the invention, it would have been obvious to one of ordinary 
skill in the arts to combine the teachings of Ralston and Kato because the interface of Kato provides accurate 
tracking and registration techniques and an intuitive and useful interface (Kato: abstract). 
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Conclusion 

9. Examiner's Remarks: Examiner has cited particular columns and line numbers in the references applied 
to the claims above for the convenience of the applicant. Although the specified citations are representative of the 
teachings of the art and are applied to specific limitations within the individual claim, other passages and figures 
may apply as well. It is respectfully requested from the applicant in preparing responses, to fully consider the 
references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the 
passage as taught by the prior art or disclosed by the Examiner. In the case of amending the claimed invention, 
Applicant is respectfully requested to indicate the portion(s) of the specification which dictate(s) the structure relied 
on for proper interpretation and also to verify and ascertain the metes and bounds of the claimed invention. 

10. Any inquiry concerning this communication or earlier communications from the examiner should be 
directed to Shambhavi Patel whose telephone number is (571) 272-5877. The examiner can normally be reached on 
Monday-Friday, 8:0.0 am - 4:30 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Kamini Shah 
can be reached on (571) 272-2279. The fax phone number for the organization where this application or proceeding 
is assigned is (571) 273-8300. 

Information regarding the status of an application may be obtained from the Patent Application Information 
Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR 
or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more 
information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the 
Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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